Rare Moments with Penalties